Vector Space Models of Lexical Meaning

Much of this Handbook is based on ideas from Formal Semantics, in which the meanings of phrases or sentences are represented in terms of set-theoretic models. The key intuition behind Formal Semantics, very roughly, is that the world is full of objects; objects have properties; and relations hold between objects. Set-theoretic models are ideal for capturing this intuition, and have been succcessful at providing formal descriptions of key elements of natural language semantics, for example quantification. 1 This approach has also proven attractive for Computational Semantics – the discipline concerned with representing , and reasoning with, the meanings of natural language utterances using a computer. One reason is that the formalisms used in the set-theoretic approaches , e.g. first-order predicate calculus, have well-defined inference mechanisms which can be implemented on a computer (Blackburn & Bos, 2005). The approach to natural language semantics taken in this chapter will be rather different, and will use a different branch of mathematics from the set theory employed in most studies in Formal Semantics, namely the mathematical framework of vector spaces and linear algebra. The attraction of using vector spaces is that they provide a natural mechanism for talking about distance and similarity, concepts from geometry. Why should a geometric approach to modelling natural language semantics be appropriate? There are many aspects of semantics, particularly lexical semantics, which require a notion of distance. For example, the meaning of the word cat is closer to the meaning of the word dog than the meaning of the word car. The modelling of such distances is now commonplace in Computational Linguistics, since many examples of language technology benefit from knowing how word meanings are related geometrically; for example, a search engine could expand the range of web pages being returned for a set of query terms by considering additional terms which are close in meaning to those in the query. The meanings of words have largely been neglected in Formal Semantics, typically being represented as atomic entities such as dog , whose interpretation is to denote some object (or set of objects) in a set-theoretic model. In this framework semantic relations among lexical items are encoded in meaning postulates, which are constraints on possible models. In this chapter the meanings of words will be represented using vectors, as part of a high-dimensional " semantic space ". The fine-grained structure of this space is provided by considering the contexts …